State-based confidence bounds for data-driven stochastic reachability using Hilbert space embeddings
نویسندگان
چکیده
In this paper, we compute finite sample bounds for data-driven approximations of the solution to stochastic reachability problems. Our approach uses a nonparametric technique known as kernel distribution embeddings, and provides probabilistic assurances safety systems in model-free manner. By implicitly embedding Markov control process reproducing Hilbert space, can approximate probabilities with arbitrary disturbances simple matrix operations inner products. We present point-based through construction confidence that are state- input-dependent. One advantage is responsive non-uniformly sampled data, meaning tighter feasible regions input-space more observations. numerically evaluate approach, demonstrate its efficacy on neural network-controlled pendulum system.
منابع مشابه
Hilbert Space Embeddings of Predictive State Representations
Predictive State Representations (PSRs) are an expressive class of models for controlled stochastic processes. PSRs represent state as a set of predictions of future observable events. Because PSRs are defined entirely in terms of observable data, statistically consistent estimates of PSR parameters can be learned efficiently by manipulating moments of observed training data. Most learning algo...
متن کاملHilbert Space Embeddings of POMDPs
A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes’ rule to these distribution embeddings. Policies and value functions are defin...
متن کاملHilbert Space Embeddings of PSRs
Many problems in machine learning and artificial intelligence involve discrete-time partially observable nonlinear dynamical systems. If the observations are discrete, then Hidden Markov Models (HMMs) (Rabiner, 1989) or, in the control setting, Partially Observable Markov Decision Processes (POMDPs) (Sondik, 1971) can be used to represent belief as a discrete distribution over latent states. Pr...
متن کاملSimulation-based confidence bounds for two-stage stochastic programs
This paper provides a rigorous asymptotic analysis and justification of upper and lower confidence bounds proposed by Dantzig and Infanger (1995) for an iterative sampling-based decomposition algorithm, introduced by Dantzig and Glynn (1990) and Infanger (1992), for solving two-stage stochastic programs. Extensions of the theory to cover use of variance reduction, different iterative sampling s...
متن کاملHilbert Space Embeddings of Hidden Markov Models
Hidden Markov Models (HMMs) are important tools for modeling sequence data. However, they are restricted to discrete latent states, and are largely restricted to Gaussian and discrete observations. And, learning algorithms for HMMs have predominantly relied on local search heuristics, with the exception of spectral methods such as those described below. We propose a nonparametric HMM that exten...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Automatica
سال: 2022
ISSN: ['1873-2836', '0005-1098']
DOI: https://doi.org/10.1016/j.automatica.2021.110146